Single-tree GMM training

In this short document, we derive a tree-independent single-tree algorithm for Gaussian mixture model training, based on a technique proposed by Moore [8]. Here, the solution we provide is tree-independent and thus will work with any type of tree and any type of traversal; this is more general than Moore’s original formulation, which was limited to mrkd-trees. This allows us to develop a flexible, generic implementation for GMM training of the type commonly found in the mlpack machine learning library [3]. A better introduction to Gaussian mixture models, their uses, and their training is given by both [9] and [2]; readers unfamiliar with GMMs should consult those references, as this minor discussion is intended as more of a refresher and also for terminology establishment. Before describing the single-tree algorithm, assume that we are given a dataset S = {p0, p1, . . . , pn}, and we wish to fit a Gaussian mixture model with m components to this data. Each component in our Gaussian mixture model θ is described as cj = (φj , μj ,Σj) for j ∈ [0,m), where φj = P (cj |θ) is the mixture weight of component j, μj is the mean of component j, and Σj is the covariance of component j. Then, the probability of a point arising from the GMM θ may be calculated as